This invention relates, in general, to processing within a computing environment, and in particular, to serializing processing in a multi-processing computing environment.
Typically, in machines based on the z/Architecture® offered by International Business Machines Corporation, Armonk, N.Y., when a program (referred to as a guest in this context) is running under LPAR (logical partition mode) or z/VM®, guest instructions are executed by the hardware or firmware. This execution of an instruction often includes translation of operand addresses and fetching or storing of data from or to those addresses. While each instruction is being executed, the hardware guarantees that any invalidate page table entry (IPTE), dynamic address translation (DAT) table entry (IDTE), or compare and swap and purge (CSP/G) requests that come in from another processor and may invalidate the translation associated with that instruction wait until the instruction is complete. Similarly, no translation or storage access is made which may be applicable to a pending IPTE, IDTE or CSP/G until that IPTE, IDTE or CSP/G operation is complete.
There are instances where the LPAR or z/VM® hypervisor needs to emulate an instruction on behalf of the guest. In order to do this, the hypervisor may need to manually translate (i.e., perform a multi-step translation unknown to the hardware) the operand addresses associated with the instruction and make subsequent storage accesses based on those translations. In order to guarantee that no conflict occurs between an IPTE, IDTE, CSP/G issued by one processor in the guest multi-processing (MP) configuration and an instruction emulation being performed by the hypervisor on behalf of another processor in that same configuration, a single interlock is used. This lock, referred to as the IPTE Interlock, resides in a System Control Area (SCA), which is a control block that is shared between all MP guests for a particular virtual configuration. The hypervisor would hold the lock during an instruction emulation and the firmware would hold the lock when executing an IPTE, IDTE or CSP/G.
To improve performance, the single lock has been split into a two-part shared lock. One part of the IPTE Interlock is a shared lock maintained by the hypervisor or host and is held by the hypervisor during emulation of an instruction; the other portion is maintained by firmware on behalf of the guest and is held while executing an IPTE, IDTE, or CSP/G. Each lock is a count of host or guest processors who currently hold the lock. To ensure consistency in the lock, the hypervisor or firmware increments the appropriate lock count (host or guest, respectively) using a compare and swap (e.g., Compare and Swap (CSG) instruction) only if the other lock (guest or host, respectively) is not held. If the host determines that the IPTE Interlock is held by the guest, it waits until the lock is available before proceeding with the instruction emulation; if the guest detects that the host lock is held, it intercepts (with an instruction interception) back to the host.
On a large single-image system, the overhead to perform the compare and swap may become substantial. When a processor is executing an IPTE, IDTE, or CSP/G, it performs a broadcast fast-quiesce operation to notify all the processors in the system of the invalidation. Only one fast-quiesce operation is allowed in the system for any one partition. If a processor issues a fast-quiesce operation and the quiesce hardware is already busy with another request, this request is rejected. On this large, single image when running a workload that is high in quiesce requests, rejections are frequent. In order to ensure proper execution, the IPTE Interlock is obtained before issuing the fast-quiesce request and then is released after the operation, even when the request is rejected. Each compare and swap causes the associated cache-line to be fetched exclusive and on a large, multi-node, single-image machine the system-wide penalty for this can be considerable.